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ABSTRACT 



As part of its contract to develop a framework for 
continuous school improvement in its four-state region (Kentucky, Tennessee, 
Virginia, and West Virginia) , Appalachia Educational Laboratory staff 
designed the Quest project. Based upon principles of inquiry, collaboration, 
and action research, Quest supports and investigates ongoing school 
improvement efforts through rallies, summer symposia, a Scholars program, 
visits to participating schools, communication via listserv and mailings, and 
the creation of a Quest network of schools. This report describes and 
assesses a high school network rally, held on February 15-16, 1999, and a 
Quest Scholars meeting held on February 14, 1999, in Roanoke, Virginia. Teams 
consisting of students, teachers, parents, and administrators from six high 
schools attended, for a total of 36 participants. At the Scholars' meeting, 
five participants and Quest staff planned a summer symposium and discussed 
how schools might collect data to document their Quest journey. Rally 
evaluation data were generated by evaluator participant observation, 
unstructured interviews, and three feedback forms. Data indicate that the 
rally's six goals- -connect with colleagues, create a learning community, 
connect with concepts related to continuous school improvement, create 
personal and shared meaning, commit to continue learning with this community, 
and commit to continue this Quest back home--were met very well. The goal 
concerning connecting with colleagues was best met; the two goals concerning 
the creation of and commitment to the Quest learning community were the least 
well achieved, perhaps because of the high number of attendees new to Quest 
events. Participants also rated specific sessions or activities concerning 
the California Protocol process, brain-based learning, student voice and 
involvement, and review of school improvement plans. Appendices present 
feedback forms and the evaluation standards checklist. Contains 18 
references . (Author/TD) 
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AEL's mission is to link the knowledge from research with the wisdom from practice to improve 
teaching and learning. AEL serves as the Regional Educational Laboratory for Kentucky, Tennessee, 
Virginia, and West Virginia. For these same four states, it operates both a Regional Technology 
Consortium and the Eisenhower Regional Consortium for Mathematics and Science Education. In 
addition, it serves as the Region IV Comprehensive Technical Assistance Center and operates the ERIC 
Clearinghouse on Rural Education and Small Schools. 

Information about AEL projects, programs, and services is available by writing or calling AEL. 




Appalachia Educational Laboratory 
Post Office Box 1348 
Charleston, West Virginia 25325-1348 
304/347-0400 
800/624-9120 (toll-free) 
304/347-0487 (Fax) 
aelinfo@ael.org 
http://www.ael.org 



This publication is based on worksponsored wholly or in part by the Office of Educational Research 
and Improvement, U. S. Department of Education, under contract number RJ96006001 . Its contents do 
not necessarily reflect the views of OERI, the Department, or any other agency of the U. S. Government. 

AEL is an Affirmative Action/Equal Opportunity Employer. 
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EXECUTIVE SUMMARY 



As part of its contract to develop a framework for continuous school improvement in its four- 
state region, Appalachia Educational Laboratory (AEL) staff designed the Quest project. Based 
upon principles of inquiry, collaboration, and action research, Quest proposes to support and 
investigate ongoing school improvement efforts through twice-yearly conferences (which staff 
renamed rallies), summer symposia, a Scholars program, visits to participating schools, 
communication via listserv and mailings, and the creation of a Quest network of schools. This 
evaluation report describes and assesses one high school network rally, convened February 15-16, 
1999, in Roanoke, Virginia. In addition, this report documents a three hour Quest Scholars meeting 
held on February 14, 1999. 

Thirty-six participants attended the rally, as well as three Quest staff and the evaluator. The 
participants included 12 students, 17 teachers, 3 assistant principals, 2 principals, and 2 parents. 
Sixteen were men, and 20 were women. Three participants were African-American, two of whom 
were mother and daughter; one participant might have been of Hispanic origin, and the remainder 
would be considered white. Attendees were asked on a final evaluation form how many prior Quest 
events they had attended. Ten were new participants and had attended no previous events, although 
they were employed by schools already in the network; nine had participated in one or two other 
Quest gatherings; and eight had been involved in three or more. Teams, ranging in size from 2 team 
members to 14, from six schools attended. The largest team included participants who might not 
become Quest team members but who were present to demonstrate use of the California Protocol 
process. 

The rally was evaluated in terms of whether, and to what extent, the conference goals were 
met. To this end, a variety of data were gathered: field notes were taken during evaluator participant 
observation of all conference activities; participants completed three feedback forms; and 
unstructured interviews were conducted throughout the rally. The rally goals included: (1) to 
connect with colleagues; (2) to create a learning community; (3) to connect with concepts related to 
continuous school improvement; (4) to create personal and shared meaning; (5) to commit to 
continue learning with the Quest community; and (6) to commit to take the Quest back home. 

Analysis of the feedback data revealed that participants thought the rally goals had been very 
well met. On a 5-point Likert-type scale, mean ratings of the degree to which goals were achieved 
ranged from 4.64 to 4.29. Most well-achieved at 4.64 (SD .56) was the goal concerning connecting 
with colleagues, and participants reported a high level of satisfaction with collegial discussions at 
the rally. Less well-achieved were the two goals concerning the creation of and commitment to the 
Quest learning community, although with identical mean ratings of 4.29 (SD’s of .71 and .76 
respectively), respondents nonetheless indicated that the goals had been met. It is possible that these 
goals were achieved less well than others because of the high number of attendees new to Quest 
events. Qualitative data tended to support the positive quantitative assessments of the rally. 




r* 



O 



li 



Based upon the data, the evaluator concluded that the rally had continued to provide support, 
encouragement, and collegial connection to those undertaking continuous school improvement. 
Recommendations included continuing to offer community-building activities, support for 
improvement efforts undertaken in network schools, and orientation sessions for new participants. 
A methodological recommendation was for Quest staff, in order to facilitate comparisons across 
rallies and networks, to disaggregate data by school level should further combined elementary and 
high school rallies be convened. 
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INTRODUCTION 



In the summer of 1996, Quest staff at the Appalachia Educational Laboratory (AEL) began 
working with teams from school communities in three West Virginia county school districts to 
invigorate efforts for continuous school improvement, using a variety of techniques for gathering 
input from all those with a stake in their local schools (Howley-Rowe, 1998g). This first “learning 
community,” called Leadership to Unify School Improvement Efforts (LUSIE), was comprised of 
school teams including students, teachers, administrators, parents, and community members. 
Ultimately, this group wrote individual school visions and improvement plans, and co-authored (with 
AEL) Creating Energy for School Improvement (1997), a supplemental guide for those poised to 
write their own state-mandated school improvement plans. 



Quest staff also were committed to creating learning communities devoted to exploring 
continuous school improvement across the AEL region of Kentucky, Tennessee, Virginia, and West 
Virginia. Hence, staff scheduled a pilot Inquiry Into Improvement conference in April 1997 for 
selected region high schools. Schools were selected in several ways. Some schools were 
recommended for the Quest experience by central office staff or school administrators. Other 
schools were asked to join Quest because they had participated in previous AEL programs. Still 
other schools were invited because Quest staff believed they were primed for the kind of 
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In October 1 997 in Roanoke, Virginia, another conference was held for designated high 
schools in AEL’s region, this time with an explicit emphasis on forming and nurturing a network of 
schools (Howley-Rowe, 1998c). A similar conference was held in Nashville, Tennessee, for 
designated region elementary schools in November 1997 (Howley-Rowe, 1998a). In order to 
facilitate the development of a Quest school network and to continue to help invigorate continuous 
school improvement efforts within network schools, staff planned a sequence of events in 1998 
following these initial conferences. Dissatisfied with the conventional and prescriptive connotation 
of “conference,” Quest staff chose to call these network meetings “rallies.” Thus, all events 
previously called conferences are now termed “rallies.” 



The high school network met a second time on February 8-10, 1998, at the Pipestem State 
Park Resort in West Virginia (Howley-Rowe, 1998d), following which the elementary school 
network participated in a rally on February 22-24, 1998, in Lexington, Kentucky (Howley-Rowe, 
1998b). During the summer, 11 network members participated in the Quest Scholars program, 
meeting at a colloquium in Charleston, West Virginia, July 16-18, 1998, to collaborate with project 
staff in ongoing efforts to conceptualize, design, and research Quest (Howley-Rowe, 1998e). And 
in August, network members and other educators in AEL’s region participated in a symposium on 
assessment of student work (Howley-Rowe, 1998f). 

Beginning their second year of network activity, Quest staff invited the elementary and high 
school networks to attend a rally together on November 2-3, 1 998, at the Glade Springs Resort, near 
Daniels, West Virginia (Howley-Rowe, 1999). Approximately half of the Quest Scholars met on 
November 1, 1998, to plan with project staff several rally activities. 
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From the high school network rally in October 1997 to the combined network rally in 
November 1 998, the Quest network contained an essentially stable membership, although there were 
differences in the number of school teams attending each event and in the frequency with which 
school teams attended gatherings, a phenomenon project staff have undertaken to study. During this 
period, Quest staff hosted seven network events. 

Scholars from the high school network met again for three hours on February 14, 1 999, prior 
to the high school network rally held February 15-16 in Roanoke, Virginia, at the Patrick Henry 
Hotel. This report describes the Scholars meeting and summarizes evaluation of the high school 
rally. 



The primary audience for this report is Quest staff. It is intended to provide information to 
staff as they make decisions about future rallies and the development of the network. In addition, 
this report will be part of an ongoing series of reports about Quest events (Howley-Rowe, 1 998a-g, 
1999). This series will document the evolution of the Quest network and the process whereby staff 
strive to enable continuous school improvement. Consequently, this report may also prove useful 
to others interested in building networks or promoting school improvement over time. 



One purpose of this report is to assess whether, and to what extent, rally goals were met. 
Although the rally goals were communicated to participants in a slightly different manner than they 
are evaluated in this document, project staff nonetheless felt their goals remained consistent across 
rallies. Thus, the six major goals discussed in earlier reports also are analyzed here. These goals are: 
(1) to connect with colleagues; (2) to create a learning community; (3) to connect with concepts 
related to continuous school improvement; (4) to create personal and shared meaning; (5) to commit 
to continue learning with the Quest community; and (6) to commit to take the Quest back home. In 
addition, the description and analysis of the rally provided in this report contribute to ongoing 
documentation of the Quest project and of the development of the Quest network. 
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METHODOLOGY 



The methods used for this evaluation component of the Quest project were primarily 
qualitative, although some quantitative data were also collected and analyzed. During the rally, the 
evaluator engaged in participant observation (Becker & Geer, 1957; Emerson, 1983;Glazer, 1972; 
Miles & Huberman, 1994; Patton, 1980), a method highly suited “for studying processes, 
relationships among people and events, the organization of people and events, continuities over time, 
and patterns” (Jorgensen, 1989, p. 12). Furthermore, consistent with the Quest paradigm, 
participant observation involves “a flexible, open-ended, opportunistic process and logic of inquiry 
through which what is studied constantly is subject to redefinition based on field experience and 
observation” (Jorgensen, 1989, p. 23). This method “is a commitment to adopt the perspective of 
those studied by sharing in their . . . experiences” (Denzin, 1989, p. 156), thereby enabling 
researchers to evaluate how an event or process appears and feels to participants. By “exploiting] 
the capacity that any social actor possesses for learning new cultures, and the objectivity to which 
this process gives rise,” participant observation further produces data that is both rich and valid 
(Hammersley & Atkinson, 1983, p. 8). 

During this rally, the evaluator played a role more akin to what Denzin (1989) typifies as 
“observer as participant,” rather than as a complete participant observer. That is, the evaluator’s 
contact with rally attendees was not as a participant in the activities in which they were engaged, but 
instead as a roaming onlooker and occasional conversationalist. The evaluator sat in on participant 
group endeavors, watched the large group as the rally unfolded, shared evening entertainment 
activities, and took advantage of serendipitous occasions to chat with participants. 

In order to corroborate the theses generated by participant observation, the evaluator also 
analyzed data from the feedback forms designed by Quest staff that solicited participant assessment 
of the process (see Appendix A). Using several data sources in order to corroborate theses is what 
Brewer and Hunter (1989) call “multimethod research” or “triangulation.” This approach posits that 
the strengths of each method will compensate for the weaknesses in others, ultimately providing a 
more complete account of that being studied. Hence, in addition to participant observation, three 
evaluation forms were used to collect further information. One feedback form asked attendees to 
discuss their experiences during the first day of the rally. This form asked participants to record: 
“Learnings, insights, ah-ha’s from the day,” “Ways in which I contributed,” “Things I want to 
explore further,” “Things that worked especially well for me,” “Things that would have allowed me 
to contribute more,” and “Things to trash.” Another feedback form was distributed at the end of the 
rally and asked participants for their evaluations of specific activities, including demonstration of 
the California Protocol, review of network schools’ improvement plans, review of brain-based 
learning research, and use of the Quest self-scoring profile. This form also included a quantitative 
assessment of the degree to which participants thought the rally goals had been achieved, using a 5- 
point Likert scale. A third form posed open-ended questions concerning specific examples of ways 
each of the rally goals had been met. 
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As a third method of data collection, unstructured interviews were conducted during the 
course of the rally. As opportunities arose for relatively private conversation, participants were 
asked to discuss their assessments of the rally in general and, more specifically, of the achievement 
of rally goals. Interview responses later were coded and analyzed by theme. 

Analyses of participant observation field notes, interview data, and evaluation questionnaire 
data were made by question, and by theme within question. Themes were coded, and responses 
within each theme were tabulated. Field notes were condensed for inclusion in the descriptive 
section of this report. 

Quantitative data were entered into an SPSS database. SPSS also was used to conduct 
descriptive analysis of the data, including calculations of means, standard deviations, range of 
ratings, and Cronbach’s Alpha, a measure of internal consistency reliability. 

Pseudonyms are used throughout this report for participant and school names in order to 
protect the anonymity and confidentiality of those involved in the Quest network. 
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SCHOLARS MEETING 



Quest Scholars belonging to the high school network met for three hours with project staff 
on Sunday, February 14, 1999. Five Scholars attended, including two school administrators, a 
teacher, a student, and a parent. One participant was male, the remainder female. One Scholar 
would be considered African-American, while the rest would be deemed white. Three staff members 
participated as well, although one arrived late due to travel problems. All sat around a table in a 
conference room in the Patrick Henry Hotel during the meeting. 

After welcoming the Scholars at 2:40 p.m., the facilitator described the agenda for the 
meeting, which included discussing the creeds prepared at earlier rallies, the school portfolios or 
logs, a research study being undertaken by Quest staff, the summer symposium, and the Scholars 
program itself. 

The group then attempted to reconstruct from memory the discussions held at the combined 
high school and elementary rally in November 1998 about a combined creed or separate creeds for 
each network. Many struggled with the reconstruction, but remembered as staff distributed a 
summary of group discussions that half had preferred one creed and half had preferred separate 
creeds. One Scholar commented, “As a secondary person, I have a hard time accepting a high school 
creed that doesn’t include caring” as the elementary creed did. Another asked, “To what extent do 
we honor the polarity, or try to revise the creeds in response to it?” Later, she added, “How will we 
use the creed over time?” The facilitator replied that creeds were intended to be a product of 
participants themselves rather than staff, and that “our hope is that it would light some fires.” 
Another staff member contributed, “It’s grounding, something to believe in.” One facilitator 
wondered whether participants would know from what level creed statements were drawn if they 
were unmarked as such. A Scholar exclaimed, “That was a great idea you had!” She then suggested 
that Quest staff print each creed statement on strips of paper without indicating from which school 
level they came, and ask network members to rate the value of each. This, the Scholars decided, 
might help ease some participants’ adherence to separate creeds. The group decided to collect data 
at the rallies using the method above, and then analyze such data at the Scholars colloquium in July 
1999. Ultimately, the Scholars felt that a combined creed was more useful than two separate 
versions. 

Next, one staff member and the evaluator discussed the research study on levels of 
engagement with the Quest network as Scholars listened quietly. They had no questions or concerns 
with regard to the study, although two briefly mentioned factors impacting engagement they felt were 
significant. A staff member provided an update on the participation of various network schools. 

Following this conversation, staff asked Scholars to set a date for the next Scholars 
colloquium; several were suggested, although none were ultimately selected, pending confirmation 
with the elementary level Scholars. The group also discussed various locations for the colloquium; 
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again, several suggestions were made, although none were selected at the time. Staff elaborated on 
the purpose of the colloquium, describing it as “primarily a writing retreat” ultimately producing 
material for a Quest book. 

During a break, the third staff member arrived and the group reconvened at 4: 1 5 p.m to talk 
about the summer symposium, to be held August 26-27 in Gatlinburg, Tennessee. A staff member 
described the symposium as a more conventional conference with various sessions. Also, the 
symposium is intended to provide network schools an opportunity to meet and learn together 
between the rallies in February and November. Although schools not part of the Quest network 
would be invited to attend, the registration fee for project teams would be waived. And finally, the 
symposium would be designed to address the interests of attendees from all four states in AEL’s 
region, as well as those falling anywhere along the K-12 continuum. 

Next, staff discussed the topics under consideration, including more theoretical sessions on 
authentic achievement and teaching for understanding. Other sessions might cover project-based 
learning, interdisciplinary teamed instruction, motivation, assessment of learning styles, and 
emotional intelligence. After describing the topics in some detail, staff asked Scholars for their 
feedback. One offered a resource for training on emotional intelligence, and another asked if the 
number of sessions offered would be based upon the number of attendees. Staff briefly replied that 
the number of participants would likely impact the sessions offered. Asked again for their feedback, 
the Scholars expressed their satisfaction with plans for the symposium. 

The next topic of discussion was how schools might collect data to document their Quest 
journey. A facilitator put it this way: “We struggled with the school portfolio or journal and other 

issues about how to show Quest made a difference for schools and especially students But we 

are real aware you have limited time [to devote to this] We want whatever you do to serve your 

needs first.” Then she distributed a worksheet to be used by school teams at the rally to link their 
own school improvement and data collection to Quest. “We hope to have teams meet and go 
through their local school improvement plans and check for goals that could be impacted or 
supported by Quest, and think about them in terms of the Quest framework,” a staff member 
elaborated. Noting that at an earlier Scholars meeting participants had suggested refraining from 
using the term “school portfolio” because of its negative connotations to school people, staff noted 
that they were calling such efforts “local school improvement logs” instead. Using the worksheet, 
“we will ask you to think about how you would document your journey ... we thought this might 
be a beginning,” added a staff member. Asked for feedback, one Scholar replied, “This will provide 
a very worthwhile focus for school teams to look at how Quest and how [sic] its methodologies 
might impact them . . . but it will be a tough task.” A staff member returned, “We don’t want this 
to be an end in itself.” Another Scholar contributed, “I’ll get to see school improvement!” One 
participant noted that documentation at her school had been a problem because they had a new 
administrator each year. Overall, however, the Scholars appeared to think the activity would be 
useful. 
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Finally, the group discussed the self-assessment profile developed by staff and the agenda 
for the rally. The conversation became more relaxed as Scholars shared information about their 
schools, other network school activities, and community events. The group dispersed at 5:40 p.m. 
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RALLY ACTIVITIES 



Day One 

The high school network rally began at 8:00 a.m., on Monday, February 15, 1999, at the 
Patrick Henry Hotel in Roanoke, Virginia, after a continental breakfast at 7:30 a.m. Thirty-six 
participants attended, as well as three Quest staff and the evaluator. The participants included 12 
students, 17 teachers, 3 assistant principals, 2 principals, and 2 parents. Sixteen were men, and 20 
were women. Three participants would be considered African- American, two of whom were mother 
and daughter; one participant might have been of Hispanic origin, and the remainder would be 
considered white. Asked at the beginning of the rally how many rallies they had attended previously, 
ten reported being new participants, four had attended one rally, and the remainder had attended at 
least two rallies thus far. Teams from six schools attended, ranging in size from 2 team members 
to 14. The largest team included participants who might not ultimately become Quest team 
members, but who were present to demonstrate use of the California Protocol process. 

The rally took place in a rectangular hotel conference room, with five round tables around 
which school teams sat. One wall had a large bank of windows letting natural light in. On each 
round table were supplies such as markers, Post-it notepads, scissors, pens, and bags full of 
manipulable toys. Around the room were stands for holding large pads of butcher paper, while along 
one side wall was a table filled with books, supplies, and other resources. At the front of the room 
were two tables arranged to create a T, containing facilitators’ supplies, an overhead projector, and 
books. On the walls, project staff had placed posters with quotes pertinent to continuous 
improvement and the Quest time line. 

Following a welcome to participants from Quest staff, each school team introduced its 
members. A facilitator then described the rally goals, which included connecting with colleagues 
and the Quest framework, learning about the Protocol process, creating a plan for a local school 
improvement log, creating personal meaning around principles of brain-based learning, learning 
about Quest network activities, and committing to continue the Quest back at school. While these 
were goals forwarded for the rally, they were based upon the six previously mentioned goals 
evaluated at prior rallies. Ultimately, staff felt the original goals remained important and ought 
therefore to be evaluated as before. 

Next, the facilitators discussed the agenda. Monday morning activities would include sharing 
success stories and learning about the Protocol process, while in the afternoon participants would 
review the Quest framework, plan for Quest back in school, and participate in focus groups for the 
research study on engagement with the project. Tuesday would begin with learning about research 
on brain-based education principles and planning for local school improvement work, while the 
afternoon would consist of reviewing the framework, previewing the summer symposium, and 
caucusing in school teams. 
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At 8:28, the facilitators quickly reviewed the Quest framework using a visual depiction and 
distributing a one page summary of the Quest project. Attendees were then asked at 8:34 to think 
of something their school had done recently that had generated some energy for school improvement, 
and to find another participant with whom they had not previously interacted. Partners were to share 
with each other a story about school improvement. Participants did so, milling about looking for 
partners and then talking together in pairs. At 8:44, staff requested that attendees decide which 
component of the Quest framework their story best reflected and then place a Post-it note with their 
name on a large graphic of the framework. Afterwards, the facilitators briefly noted that six or seven 
Post-it notes each had been placed on the components of assessing and demonstrating student 
learning, broadening the learning community, and strengthening the learning culture. Two Post-it 
notes were placed on the component of enabling SMART (a Quest acronym for Successful, 
Motivated, Autonomous, Responsible, and Thoughtful) learners, and one on sharing leadership. At 
8:49, the facilitators distributed large blank cards on which attendees were to write the stories they 
had shared in pairs. The room became quiet as participants did so. Staff later collected these cards. 



At 8:56, a facilitator requested that attendees return to their school teams and discuss the 
events they had chosen to share with their partners. Following several minutes of discussion, a 
representative from each team was asked to sign the Quest time line and write a brief description of 
recent accomplishments. By 9:05, most teams had done so, and the facilitators requested that each 
team report to the whole group their recent activities. The first team reported that their version of 
the Olympics, held to reward improved performance on the state standardized test, had been a “huge 
success.” In addition, their school had been wired for Internet connection with the help of parents, 
their Quest team had been growing, the school began its first semester on the 4x4 block schedule, 
and a leadership class had been successful, with students doing much of the work in preparation for 
the school Olympics. One team member ended by saying, “For first timers [to Quest], we welcome 
you because this is a great place to exchange ideas . . . things have been done at our school because 
of Quest and the monthly Quest meeting.” A second team reported that they had completed a school 
improvement plan. In addition, the school’s English department had begun using Paideia. “We 
thought we were going to a block scheduling conference, but when we got there it was Paideia . . . 
but it was wonderful! It has helped solve problems with class discussion ... it eliminates eager 
beavers dominating . . . discussion is more meaningful.” Another school described themselves as 
a very rural independent district. They noted having plenty of funding unencumbered by a local 
school district bureaucracy, and spoke of having increased their SAT scores. As for Quest, they had 
not been very involved, but as one teacher put it, “One day the superintendent showed up at my door 
and said ‘You need to go to Roanoke.’ And I said, ‘Yes, I need to go to Roanoke.’” Other Quest 
participants laughed at this. In sum, he said, “We plan, as I understand it, to be more involved,” 
amidst further laughter. 

A new superintendent had focused on technology and enabled most classrooms to be 
networked at a fourth school. In December, the school had a staff meeting at which Interview 
Design had been used with teachers and students. A team member reported that one student later 
commented, “I didn’t know teachers talked about this stuff. I thought all they did was teach.” The 
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team member summed up, “It was good to share leadership.” The Data in a Day process had been 
facilitated by Quest staff at a fifth school for the purpose of assessing effectiveness of the 4x4 block 
schedule there. In addition, the school’s parent organization had placed new welcoming signs 
around the school and the English department had collaborated using the Protocol process to address 
a new district requirement for students. And at a sixth school, the Quest team had collaborated with 
other groups to facilitate a day of interdisciplinary instruction around a theme across school 
departments. Also, the Quest team had grown, was meeting more often, and included five students. 
The school was preparing to submit a proposal for a large grant, attempting to implement a self- 
assessment strategy learned from another Quest school, and increasing the number of dual 
enrollment courses as suggested by the Quest co-venture team. 

Attendees took a break from 9:25 to 9:44 a.m. Once participants had returned to the 
conference room, facilitators distributed packets of information on the Protocol process. The 
handouts described the various roles of participants in the process, the types of feedback to be 
elicited, and how the process was to be conducted. Staff described the process to participants as 
well. 



At 10:00, six members of the Xavier Senior High School 1 team convened in front of the 
whole group to demonstrate the process. One member acted as facilitator, while the remaining five 
were analysts or presenters. The audience assumed the role of reactors. The analysts first gave some 
background concerning their topic of discussion, the mandated completion of a research paper by 
all 1 1th graders in their school district. Following this, the analysts were asked to discuss amongst 
themselves how the first round of research papers and their assessment by the district had gone. 
After approximately 20 minutes of such discussion, the reactors were asked if they would like to 
pose any questions that might clarify their understanding of the topic. Several asked questions, and 
received replies. The reactors were then asked at 10:45 to discuss at their tables what warm (or 
supportive) and cool (or critical) feedback they had for the analysts. The analysts roamed the room, 
listening in as groups discussed their feedback. At 1 1 :01, after most groups had generated lists of 
warm and cool feedback, the analysts were asked to reconvene and reflect on what they had heard. 
The team talked freely, sometimes laughing, and not appearing to have taken umbrage at any of the 
feedback they had received. Several comments indicated that team members might take action based 
on suggestions made by the reactors. 



At 1 1 :07, the facilitators asked participants to debrief their experience. One said the process 
felt comfortable, although she noted that she was more candid when participating now because she 
had been through the process before. “You settle into it as you get to know it,” she added. A student 
reported that he preferred a less structured process for discussion, while a few other disagreed. 
Another participant noted that “time was short for generating comments and for sharing [them],” to 
which the team member who had facilitated the process responded that if they had demonstrated the 
process in full, more time would have been allotted to the creation and sharing of feedback. Other 



'Pseudonyms are used throughout this report in place of participant and school names in 
order to protect the anonymity and confidentiality of those involved in the Quest network. 
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comments included: “I would have preferred a short report from each group”; “I felt the content was 
secondary to learning about the process”; “The feedback was not personal”; “It’s good to see the 
process. I like the structure ... I made headway in understanding what they’re facing because of the 
structure”; and “It felt good to understand what they’re going through.” 

A participant asked the presenting team what the “end of the story” was. The team facilitator 
responded, noting that, for instance, there was increased professionalism in the English department 
as a result of using the Protocol process. 



Following a short break between 1 1 :20 and 1 1 :30, school teams discussed how they might 
be able to use the process in their own schools. At 1 1 :44, they were asked to make brief reports to 
the whole group. One school had not found a focus for the process, but felt that it could be used 
school wide to enhance student leadership because it is “a structured arena to get something done.” 
Another school thought the process could be used to align the secondary disciplines with the 
elementary grades in order to address Virginia’s Standards of Learning. Students on the team spoke 
of using it in their classrooms, or with departments as analysts and students as reflectors. A third 
school thought they could apply the process to examining their implementation of the block schedule 
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of issues, from block scheduling to class size, while another considered using it to discuss the 
effectiveness of the freshman seminar. Students from this school thought they might apply it to 
assessing student council activities. Virginia’s Standards of Learning, detention issues, and the block 
schedule were areas a final team thought they might examine using the Protocol process. The team 
also reported considering using the modified focus group technique of which they had learned at an 
earlier rally to “get emotions out,” and then using the Protocol process to examine issues “with less 
emotionality.” 



Following these reports, participants were excused at noon for a group lunch. Returning at 
1 :00 p.m., attendees were asked to reconvene in their storytelling pairs from earlier in the morning 
and get to know each other on a more personal level using prompts from the Quest calendar that had 
been provided to all. At 1 : 1 0, a staff member discussed the co- ventures — what they were, how many 
had been completed, and of what benefit they might be to schools. 

Next, a facilitator requested that attendees complete a self-scoring profile developed by staff 
to assess the presence of Quest framework components in schools. Because the profile was still 
under development, staff asked that respondents circle vague items and underline words or phrases 
that were unclear. Participants quietly completed the instrument until 1 :32 p.m., when they were 
then asked to convene in role-alike groups to discuss the profile collectively. Items of discussion 
were to be what questions did not “speak” to each group, what concepts were missing from the 
profile, and “what is the potential value of this for your peers?” Group consensus was to be recorded 
on another copy of the profile on different-colored paper. As groups talked, the facilitators roamed 
around the room, made clarifications, and answered questions. 
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At 1:59, staff asked that each group make a brief report to the whole group about their 
discussion. A parent noted their group felt that very little on the instrument “spoke” to parents, and 
that much of it needed to “be in more ordinary language.” They also noted that many of the items 
concerned issues that parents would “have no way of knowing about.” A teacher group reported, 
“If some teachers aren’t comfortable with the words [in the instrument], then neither will be parents 
and students.” The group also noted that none of its members had felt comfortable making 
judgements about the whole school, though they added that perhaps the profile could be used to 
focus on challenges and their solutions. A representative from a student group offered that project 
staff ought to “make sure it’s given to students who really care about school improvement . . . rather 
than apathetic students rating 'O' on all [items] because they don’t like school.” The profile might 
be “a good way to reach out to student government” or to “help determine curriculum.” A second 
group of students felt the instrument could be used to examine the success of the 4x4 block schedule 
and to help with the pursual of school goals. Too, this group asked that names of respondents be 
kept “unanimous,” a comment receiving much laughter and correction. Administrators felt that item 
26 was “double-barreled,” and that an item concerning innovation should be added. The profile 
could be used to identify areas where respondents were not in agreement about the school, thought 
the administrators. Another teacher group suggested that the day on which the instrument would be 
administered ought to be chosen carefully because events or activities might “skew responses.” 
Nonetheless, they felt the profile would be useful to “get an idea where you want to go” with regard 
to school goals. 

After a brief physically energizing activity, school teams reconvened at 2:17. Teams were 
instructed to analyze their school improvement plans, which they had brought to the rally, and 
identify one or two goals or objectives that could be supported by Quest activities. A worksheet was 
distributed to assist in this process, which asked teams to name their local school improvement goal, 
the Quest component(s) to which it was related, current and future Quest activities that support the 
goal, expected results from such actions, data to document achievement of the goal, and basic 
components of a data collection plan. Beginning at 2:25 p.m., teams examined their plans, discussed 
them, and completed the worksheet. Staff requested at 2:54 that participants bring their work to 
conclusion. 

At 2:58, staff previewed the next day’s activities and then asked that teams provide brief 
reports on their Quest-supported local school improvement goals. One team said their goal was for 
all students to become responsible members of their families, community, and school, a goal they 
felt was related to every component of the framework. A second school aimed for open 
communication in an atmosphere of increased expectations. They believed this goal was related to 
the component of broadening the learning community. The Quest framework component of shared 
leadership was congruent with another school’s goal to increase the variety of students involved in 
school advisory committees. The component of broadening the learning community coincided with 
one school’s goal to consolidate two faculties for the purpose of increasing student, parent, and 
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community involvement in the school. A fifth school’s goal of increasing the percentage of its 
students passing the state competency test was supported by the component concerning assessment 
and demonstration of student learning. And finally, another school planned to increase student 
problem-solving skills, a goal they thought was related to the component of enabling SMART 
learners. 

Between 3:00 and 3:20 p.m., staff distributed consent forms for participation in research, 
explained the research study on engagement with Quest, and divided the participants who had 
attended at least one other rally into focus groups. Those new to the network met in a small group 
with one staff member to receive a more thorough orientation to the project while the remainder 
participated in focus groups for the engagement study. Participants adjourned for the evening by 
approximately 4:10 p.m. after completing the first evaluation form. 



Day Two 

Following a continental breakfast in the hotel lobby, the rally reconvened at 8:08 a.m. on 
Tuesday, February 1 6, 1 999. One participant brought a compact disc player for Quest staff, who had 
forgotten to pack one earlier, to borrow during the remainder of the rally. Music played quietly in 
the background. One participant took photos of her team. At 8:10, the facilitators welcomed 
attendees back and asked about their evening activities. Next, they announced the summer 
symposium and described its proposed content. 

A facilitator then asked if participants had any comments concerning the previous day’s 
activities. A principal noted that students had ideas for using Quest; she said she was “curious if 
they’d be interested in leading a session” at the symposium. A staff member replied, speaking 
directly to students, “if there’s a special interest you have, that you could gather your peers for” space 
could be made available for such a session. Another participant commented that parents in a 
discussion group the previous day had reported “really wanting] to be useful.” That night, the 
participant and another team member had compiled a list of ways parents at their school could 
become more involved. The attendee further suggested instituting a “walkabout,” an evaluative 
process shared by another team at an earlier rally, including parents to examine structural problems 
at their school with which they might assist in repairing. As a means to continue discussing parent 
involvement, a facilitator requested that one participant describe how Data in a Day was used at her 
school incorporating the perspectives of parents, students, and staff. 

Following the discussion of Data in a Day, the facilitators described the day’s agenda and 
then asked attendees to number off from one to six to form six groups. These groups were assigned 
one component each of the Quest framework. Their “task is to think about your piece of the 
framework, what it means for a high school, and what a high school might look like if it had a strong 
learning culture [for example] . . . then present some kind of visual display representing [your 
discussion].” Groups were allotted 30 minutes to complete the activity and take a break. 
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Groups began discussing their assigned component, often sharing current practice at their 
schools with regard to the component under consideration. Quest staff walked around the conference 
room, listening to discussions and answering questions. Although groups worked diligently, talking 
and creating posters, by 9: 1 5 many had not yet taken their break. The facilitators allowed 1 5 more 
minutes for the activity. 



By 9:37, most participants had returned from their break, and at 9:40 Quest staff reconvened 
the entire group. A student began by reporting for the group considering learning culture. He said 
teachers ought to be chosen carefully (a comment receiving laughter) and that pep rallies and sports 
teams helped build a sense of learning community, as did academic and cultural clubs. The group 
discussing shared leadership reported that a high school with such leadership would have clubs with 
officers and faculty advisors, department chairs and administrators would be in close 
communication, student government would be student driven, and “open communication without 
retribution” would be the norm. In addition, school community members would speak more 
frequently of “we” than “I.” A third group read aloud their “Ode to the Learning Community,” a 
poem filled with rhymes that drew hoots of laughter and applause. 



A 



picture of a hand represented the discussion of shared goals for student learning in the 
fourth group. A participant reported their notion that goals ought to be simple and in accessible 
language; hence, their depiction of a goal in the palm of the hand, with fingers representing 
constituent objectives and stakeholders. Goals, the group continued, ought to be established by 
consensus of the school community and ought to focus on desired results rather than the mechanisms 
by which they might be achieved. The acronym SMART was broken down by letter and explicated 
by another group. Their poster read: “For Success, a program must be diverse enough so every 
student finds a niche — athletics, vocational, academics, arts, special needs, gifted, work programs. 
For Motivation, positive atmosphere, positive reinforcement, excitement, home support, goal setting. 
For Autonomy, freedom to take initiative, student choice in assignments, freedom to find own 
answers. For Responsibility, Success + Autonomy = Responsibility. Provide consequences, high 
expectations, accountability. For Thoughtful, higher level thinking skills, thought-centered as 
opposed to knowledge-centered, citizenship.” A final group presented a skit of a student learning 
to juggle and a teacher offering support through assessment. They hypothesized three steps: (1) 
learning, aided by use of diagnostic quizzes or tests; (2) application, aided by lab projects and 
research papers; and (3) learning as a lifelong process, aided by learning how to learn. The final 
product, they noted, would be a successful member of the community. 



At 1 0:00, the facilitators thanked participants for their work, discussed their hopes to produce 
a book about Quest, and invited attendees to apply to the Scholars program. At 10:04, staff began 
to discuss the morning’s major topic, brain-based learning. A facilitator pointed to several objects 
on tables that were important to enhancing brain functioning. These included water, nuts, plants, 
and music. After sharing several resources about brain research and teaching strategies based on 
findings from such research, staff presented “fast facts” about the brain. Next, attendees were asked 
to convene in role-alike groups to read scenarios “embody[ing] much about contexts that support 
brain-based principles.” Teachers were to read a piece about a classroom, parents about a home, 
administrators about a school, and students about a student aware of brain-based principles. After 
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reading their particular scenarios, role-alike groups were to identify the factors or strategies described 
that were drawn from brain research. From this, groups were to develop lists of brain-based 
principles and discuss the degree to which each component was common practice. Facilitators 
allotted 45 minutes for the activity. 

At 10:16, role-alike groups formed, although some attendees went to the hotel lobby and 
checked out of their rooms. One of the parent participants did not return for the second day of the 
rally, leaving only one parent. A facilitator volunteered to participate in the activity as a parent, and 
she asked if any participants would join her. One teacher agreed, making a group of three. Some 
groups met in the hotel lobby, while most students went to their rooms to gather belongings. By 
10:30, the group of administrators had completed their discussion. Teachers continued to talk, while 
the facilitators conferred with groups as they had questions or concerns. One school team of four 
left the rally in order to return for a sports event at their school, and by 1 1 :00 some groups had not 
yet finished with the activity. 

At 11:14, Quest staff hung up four large pieces of butcher paper, each representing a 
category: environmental, physiological, psychological, and instructional. A facilitator then began 
a whole group discussion of the principles described in the scenarios, aiming to place each in one 
of the four categories. Participants offered their conclusions as a project staff member created a 
Mindmap on each piece of paper representing the discussion. On the paper representing 
environmental factors, topics included the impact of aromas, music, parental support, 
communication, positive classroom atmosphere, and celebrations on learning. Under physiological 
factors were the importance of exercise, nutrition, and rest to full brain functioning. Psychological 
issues included self-motivation, goals, high standards, knowing oneself, and enthusiasm. 
Instructional strategies listed included multisensory learning, self-assessment, use of aromas in the 
classroom, and making meaning. 

At 1 1 :55, a facilitator summed up the activity and discussed in more detail a point made by 
a participant concerning the importance of making meaning, “connecting content and personal 
interest, experience.” Attendees were dismissed for lunch at 12:07. 

Participants reconvened at 1 :05 p.m. and were asked to discuss in school teams how they 
might use their learnings about brain-based principles. At 1 :25, groups were requested to share their 
discussions. One team considered planning a theme day for faculty around brain-based learning, 
then requiring that teachers plan to use at least some brain-based principles and tie them to the state’s 
learning standards. A second school team shared that they had participated in two days of training 
on the subject. As a result they had repainted the school walls in more neutral colors, installed new 
lighting, and brought plants into classrooms. They reported being “basically on track.” Another 
school reported being in the midst of renovations; they hoped to have more plants throughout the 
school and “get rid of those ‘60's’ colors.” Repainting in cooler colors, replacing old lighting, and 
placing couches in classrooms were goals of another school team. A final school noted that they 
hoped to have a “Take Pride in Your School Day,” during which improvements based on brain 
research findings might be made. They also noted having painted over their school’s orange lockers. 
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A participant commented on the presence of natural light, reported to optimize brain functioning, 
during the rally itself, noting that the windows in the hotel conference room “probably [are] making 
the conference better.” Several others nodded in agreement. 

Teams were next asked to spend five minutes discussing a topic they wanted to present using 
the Protocol process with another school team. At 1 :40, project staff quickly reiterated the process 
and provided guidelines for how long each component might last. Each team was allotted 
approximately 35 minutes to complete the process before the team previously taking the role of 
reactors took a turn as presenters. In this way, each team would have chance to present and receive 
feedback. 

At 1 :45, however, an impromptu performance by two participants took place after others 
encouraged them to do so. A teacher played a piano in the hotel lobby while a student sang 
“Yesterday” by the Beatles, without time to practice beforehand. The group gathered around the two 
and listened with apparent appreciation. Afterward, they applauded the performers loudly. 

Back in the conference room, pairs of teams met to participate together in the Protocol 
process. One team, however, was unable to stay for the remainder of the rally; only four teams 
remained. A facilitator sat with each pair, listening and often reminding participants of the process 
guidelines. Attendees listened to each other, although quite a few appeared tired or restless, gazing 
across the room or fiddling with toys. At 2:32, the first teams to present finished, and then took their 
turn as reactors for the second teams. Again, attendees participated, listening and offering feedback 
as the process called for, but several seemed tired or unenthusiastic. 

At 3 :05, facilitators asked participants for their comments about howthe process had worked. 
A student commented that he preferred the modified focus group of which he had learned at an 
earlier rally, because it was less structured and more conducive to “back and forth discussion.” 
Another attendee thought that “in Protocol you really need to examine student work,” while another 
said simply that she “liked talking to other schools.” One said, “We didn’t have time to diagnose 
our problems. We would have planned [for Protocol] for a few weeks.” A facilitator then asked, 
“Is this kind of exchange between schools useful?” About ten participants answered “Yes” aloud. 
One commented that the Protocol process “is a good reality check,” while another termed it “an 
outside view ... no hidden agendas.” “You learn you’re not alone, that you have the same types of 
challenges.” 

Next, attendees were requested to form triads in order to complete an activity to evaluate the 
construct validity of the Quest self-assessment profile. Triads were to discuss each item and then 
place it in the framework category they felt it best represented. 

At 3:30, attendees completed two final evaluation forms and said goodbye in teams or 
individually to the facilitators. Quest staff briefly thanked all who had participated. Most 
participants had left by 3:45 p.m. 
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FINDINGS 



Achievement of Rally Goals 

The extent to which rally goals were achieved at the February high school network rally is 
discussed in the following section. The rally goals included: (1) to connect with colleagues; (2) to 
create a learning community; (3) to connect with concepts related to continuous school 
improvement; (4) to create personal and shared meaning; (5) to commit to continue learning with 
the Quest community; and (6) to commit to take the Quest back home. As noted earlier, project staff 
communicated slightly altered rally goals to attendees, but chose to evaluate the achievement of the 
original six goals listed above. Data for such evaluation are drawn from participant observation, two 
open-ended feedback forms, one form soliciting both qualitative and Likert-type quantitative 
feedback, and informal interviews. 

Participants were asked to complete three feedback forms at the rally. Thirty -two of the 36 
attendees completed the first form, distributed at the end of the first day’s activities, representing a 
return rate of 89%. Two forms were distributed at the conclusion of the rally. One form posed only 
open-ended queries; 27 participants completed this, for a return rate of 75%. The second posed both 
qualitative and quantitative questions; 28 completed this questionnaire, representing a return rate of 
78%. Response rates were likely impacted by the early departure of two teams. The open-ended 
final feedback form asked participants to denote their role. There were 14 teachers, 8 students, 4 
school administrators, and 1 student. Too, attendees were asked how many prior Quest events they 
had attended. Ten were new participants and had attended no previous events, although they were 
employed by schools already in the network; nine had participated in one or two other Quest 
gatherings; and eight had been involved in three or more. 

Participants were asked on one of the two final evaluation forms to rate the degree to which 
each of the six rally goals had been met, using a 5-point Likert-type rating scale offering response 
options from 1 (“not accomplished”) to 5 (“extremely well accomplished”). Responses received 
ranged from a low of 3 to a high of 5. Using Cronbach’s Alpha, the internal consistency reliability 
was calculated to be .79. 

Goal One, connecting with colleagues, received the highest mean rating of 4.64 and the 
second lowest standard deviation of .56, indicating that participants thought it had been well 
achieved (see Table 1). Least highly rated with identical means of 4.29 were Goal Two, creating a 
learning community, with a mean of 4.29, and Goal Five, committing to continue learning with the 
Quest community, also with a mean of 4.29, with respective standard deviations of .71 and .76. 
Nonetheless, even these two means indicated that respondents felt the two goals had been 
accomplished. 
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Table 1 



Participant Rating of the Achievement of Rally Goals 





N 


Mean 


SD* 


Goal One: Connect with colleagues 


28 


4.64 


.56 


Goal Two: Create a learning community 


28 


4.29 


.71 


Goal Three: Connect with concepts concerning school improvement 


28 


4.39 


.63 


Goal Four: Create personal and shared meaning 


27 


4.33 


.68 


Goal Five: Commit to continue learning with this community 


28 


4.29 


.76 


Goal Six: Commit to continue the Quest back home 


28 


4.32 


.55 



* Standard Deviation 

Goal One: Connect with Colleagues 

As noted earlier, the achievement of this goal received the highest mean rating (4.64, SD .56) 
from participants on the final evaluation form soliciting quantitative assessments of how well 
participants felt it had been achieved. Asked on the other final evaluation form in what ways they 
had connected with other participants, one respondent did not reply, six provided answers containing 
multiple themes, and the remainder offered answers with one theme. Eleven responses indicated that 
participants had connected with each other through the sharing of their stories, situations, and 
dilemmas. Eleven more responses suggested that connection had occurred as attendees met new 
people, while five respondents noted that they had connected with others through sharing new ideas. 
Two noted connecting with colleagues because of the continuity of those attending network 
gatherings. Other replies were idiosyncratic. One response indicated that connection occurred 
through reaffirmation of her efforts: “[I] realized there are good results when emphasis is 
put/channeled in the right direction.” Another participant connected with colleagues by discovering 
that all in attendance were “dedicated to learning.” A third respondent noted the value of small 
group work to connecting with others, while yet another appreciated the time allotted to team 
planning. 

Informal interviews also revealed that some attendees found the goal of connecting with 
colleagues to have been achieved well. As one new network member put it, “I don’t usually feel 
comfortable with strange people, but I’ve felt comfortable really fast here.” Another interviewee 
commented on the value of spending time in discussion with administrators from her school. “One 
good thing,” she said, “is that we have two administrators here . . . yes . . . and forget that they’re 
administrators, they’re both good people and they care about kids . . . that’s a good place to start 
from.” 
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Participant observation likewise revealed that participants connected well with each other 
during the rally, engaging in respectful and sometimes enthusiastic discussion with each other. In 
addition, applause and laughter seemed to indicate that attendees were attentive to and appreciative 
of one another. 

Goal Two: Create a Learning Community 

The achievement of this goal received one of the two lowest mean ratings from participants, 
with a mean of 4.29 (SD .71). Nevertheless, the rating is above the scale point 4, indicating that 
most respondents felt the goal had been achieved. 

Attendees were asked on the final open-ended evaluation form whether and by what means 
Quest network members became more of a learning community during the rally. One respondent 
did not answer this query, while one other provided an answer containing multiple themes. Twenty- 
five respondents indicated that Quest network members had become more of a learning community, 
with 23 also offering specific examples of how this happened. Eleven believed enhancement of the 
learning community occurred due to the discussions and sharing of ideas within an emotionally and 
intellectually safe atmosphere. “Ideas were shared without fear of retribution” and “the 
administrators got to play a non-threatening role, too,” were the ways in which two such respondents 
phrased this notion. The diversity of perspectives and experiences represented by participants was 
thought by four respondents to be most nurturant of the learning community, while three others 
praised the use of the Protocol process specifically. Two participants offered that the opportunities 
for school team work had been beneficial to the learning community. Three replies were unique. 
One indicated that the continuity of those attending rallies aided the development of the Quest 
learning community. Similarly, another felt that the “bonding and building of relationships” was 
most influential. A third response indicated that learning together facilitated the learning 
community. One reply, however, was somewhat more ambiguous and critical: “I think the students 
connected. I did not feel as connected to the teachers/administrators.” 

An administrator noted in an informal interview that this rally had been especially good 
because so many students attended. “There are lots of personal positives . . . being close, I could 
bring more students ... I have learned what their concerns are. I liked that.” About Quest events 
in general, one participant commented, “You get more than you give.” Too, the comment cited 
earlier, “I don’t usually feel comfortable with strange people, but I’ve felt comfortable really fast 
here,” perhaps implies that network gatherings do not feel exclusive or hostile to new participants. 
These comments together suggest that some network members found the experience of learning 
together in a community valuable and unthreatening. 

It could be argued that several occurrences enhanced the development of the Quest learning 
community during the rally. For example, near the end of the rally, participants convinced two 
network members to perform a song. An impromptu event, it appeared to bring attendees together 
for a few moments in appreciation of music and each others’ talents. One could also make the case 
that unsolicited attendee interest in the “end of the story” presented by one school team during 
demonstration of the Protocol process indicates community rather than perfunctory participation. 
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It might also be argued that the willingness of network members to critique the Quest self-scoring 
profile indicates that the rally offered participants an intellectually and emotionally safe atmosphere 
in which to learn and debate. And finally, the Quest learning community was broadened 
significantly at the rally, with the attendance of 10 students, more than had attended earlier rallies. 

On the other hand, some participants seemed restless or inattentive during the Protocol 
process near the end of the second day of the rally. As one respondent said on a feedback form, this 
may have been due to the large number of participants in the process, impeding engaged and serious 
interaction. Nonetheless, attendees’ lack of attention did not contribute to nurturing the Quest 
learning community. 

Asked on the feedback form distributed after the first day of the rally in what ways they had 
contributed, 28 of the 32 respondents wrote of having both expressed their thoughts and listened 
attentively to others do the same. The remaining four respondents did not answer the query. 

On the same form, attendees were also asked what might have allowed them to contribute 
more. Fifteen respondents did not reply to this question, perhaps because they felt they had 
contributed enough or because they believed nothing could have enhanced their contribution. Nine 
of the 1 7 who did reply reported that more time for discussion in small groups or in school teams 
would have been beneficial. Two requested more “hands-on” activities, while other comments were 
unique. One respondent felt more information about how to put the Protocol process to work would 
have enhanced her contributions, while for another a larger, more diverse group of schools would 
have done so. Student participation in groups other than school teams would have increased the 
contribution from another attendee, and yet another wrote that receiving more about Quest prior to 
the rally would have been helpful. One respondent thought he would have contributed more “if I 
would have opened my mouth.” Finally, another respondent offered that, “The group was too large 
for the Protocol demo [sic]. We either needed to be in smaller groups, or reflectors should have been 
chosen to participate.” 

Goal Three: Connect with Concepts Concerning Continuous School Improvement 

With a mean rating of 4.39 (SD .63), respondents indicated that this goal had been met. 
When asked on the open-ended final feedback form with what content related to continuous school 
improvement they had connected, nine participants reported learning about the Protocol process as 
a vehicle to discuss improvement concerns. Six responses indicated the value of learning about 
brain-based research at the rally, and three reported connecting with notions of student voice and 
involvement. Two responses each cited connection with reviews of school plans and school team 
work on communication issues. Four replies were idiosyncratic, with one noting the value of 
discussions about block scheduling and another the value of group discussions. One participant 
replied generically of having “learned from other different schools,” and another reported being 
unsure how to respond to the query. Four respondents provided answers containing multiple themes, 
and five did not reply to the question at all. 
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Attendees were asked to assess how presentations on four topics had “worked” for them on 
a second final feedback form. Sixteen reported that the time devoted to learning about the Protocol 
process had worked very well. Comments about the Protocol process included: “[It] fostered 
thinking toward improving the entire learning environment by its use”; “[Worked] very well. Plan 
to use”; and “I was very impressed with this demonstration. Our group has already thought of a 
number of ways to use this procedure.” Five participants reported that the sessions on the Protocol 
process had worked moderately well, four of whom noted that they better understood the process 
after participating in it themselves and after receiving additional explanations. On the other hand, 
four respondents thought the sessions on the process had not worked well. “[I] did not like it much; 
prefer open communication,” said one such attendee. Another reported that “[it] wasn’t exactly on 
task,” while a third found it to be “confusing.” Two comments did not appear to address the 
question posed, and one respondent did not answer this query. 

A feedback form was distributed following the first day’s activities on which participants 
were asked what “learnings, insights, ah-ha’s” they might have gained during the rally thus far. Of 
the 32 respondents, half (16) mentioned having learned of how to use the Protocol process, as one 
put it, as “an organized way for people to discuss together and achieve a common goal.” The 
remaining 16 respondents reported having gained insights regarding Quest (e.g., “Quest’s goals and 
purposes” and “I know absolutely nothing about Quest, so everything was informative for me”) and 
aspects of their own and others’ situations (e.g., “Working in a group of parents, I realized how out- 
of-touch they feel” and “I realized that I am more involved in my school than I thought”). 

When asked how the segment on brain-based learning had worked, respondents 
overwhelmingly expressed their approval. Twenty-four participants reported that learning about 
brain-based research had been beneficial. Comments included, “excellent”; “It was very interesting 
to learn about this topic. I especially liked the ‘scent’ part of brain-based learning”; and “interesting, 
enlightening.” Three respondents were more ambivalent about the topic. One such participant 
thought it had not worked as well as others because “people kept leaving,” while another 
commented, “This was interesting but somewhat vague. You did whet my appetite for learning 
more.” A final respondent simply noted having participated in professional development on the 
topic earlier. 

Attendees also reported that the review of their local school improvement plans for the 
purpose of aligning goals with Quest activities had worked well. Twenty-one respondents noted the 
value of this segment, commenting, for instance, “[It] brought some good discussion among our 
group on a problem that we are currently trying to solve”; “I like knowing what is going to be done 
and how to work towards improving the school”; and “Very helpful to our team. Helped us identify 
solutions.” One participant thought the review of school plans had been useful but that not enough 
time had been allotted for the activity. On the other hand, three respondents thought the review had 
not worked well, two because not enough time had been provided and one because it was “not 
focused.” Two comments did not appear to address the question, and one respondent did not reply 
to the query. 
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Respondents reported that activities involving the Quest self-scoring profile had worked least 
well, although 14 of those responding assessed it favorably. Three comments indicated that 
participants were ambivalent about the use of the self-scoring profile. For example, one such 
respondent wrote that it was “really a pain, but good for you. We’re happy to help.” Two 
respondents assessed the use of the self-scoring profile negatively. One attendee, for instance, 
simply “did not like” the profile. Nine respondents did not reply to this query. 

While participants were not queried directly about how well they had connected with the 
Quest framework, most appeared to enjoy the framework review activity on the second day of the 
rally. Not only did attendees create amusing presentations of framework components, such 
presentations sometimes explicated implications of the framework. For instance, the group who 
discussed the SMART learner component elaborated on the meaning of each part of the SMART 
acronym. 

Goal Four: Create Personal and Shared Meaning 

With a mean rating of 4.33 (SD .68) on the final evaluation form soliciting quantitative 
assessment, respondents indicated that the goal of creating personal and shared meaning had been 
met during the rally. Responses and nonresponses to the open-ended question concerning this goal, 
however, suggested that participants were perhaps uncertain what this goal meant or how its 
achievement might be assessed. 

On the final open-ended feedback form, participants were asked, “If you were able to create 
personal and shared meanings at this rally, what were two or three of these meanings?” Fourteen 
respondents did not answer, representing the highest rate of nonresponse on this form, perhaps 
indicating their confusion about the query or their reluctance to share reflection on this matter. Three 
other participants did write explicitly that they did not understand the question. On the other hand, 
ten respondents did answer, five of whom provided answers suggesting multiple themes. Four 
responses indicated that meaning had been created during discussions of ideas, while four more 
described the meaningfulness of discussions about brain-based learning. Two replies suggested that 
meaning had been generated around the use of the Protocol process. Other responses were unique, 
including one concerning Quest itself : “Our team was expanded, allowing for greater understanding 
of Quest and continuous school improvement.” Another participant wrote of reaffirming that 
“student learning comes first. [The] entire community needs to respond.” “The idea of a proper 
learning environment; the idea of how to engage students; the idea of how to effect positive change,” 
were cited as meanings created by yet another participant. Finally, one respondent wrote, “[Our] 
group found a resolution to a particular problem at our school.” 

Likewise, a participant in an informal interview cited the difficulty of reflecting on one’s 
practice during school hours and the consequent value of attending Quest events at which such 
reflection was central. As he put it, “The rallies are useful . . . because there’s no time to think and 
reflect on what you believe when you’re busy serving 1200 students and 100 teachers.” He later 
added that holding Quest events far from his school added to his sense of reflective remove from the 
rush of school activity: “If I were gone [out of town], people would know I was gone and couldn’t 
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do those things . . . there, I’m distracted.” Although this attendee was not replying to a query 
concerning the creation of meaning, his comments suggest that the retreat-like atmosphere and time 
provided for reflection enhanced meaning making at rallies. 

Goal Five: Commit to Learning with this Community 

The achievement of this goal received one of the two lowest mean ratings from participants 
and the highest standard deviation. However, at 4.29 (SD .76), this goal nonetheless appears to have 
been met during the rally. 

Attendees were requested via the open-ended feedback form to describe the extent to which 
they felt committed to continuing to learn with the Quest community and the reasons for the level 
of their commitment. Responses indicated a variety of commitment levels, from those who felt 
highly committed to those who were more ambivalent about their dedication. Twelve respondents 
did not reply to this question. One said simply that it was difficult to be away from school, perhaps 
indicating her struggle with commitment to Quest. Two participants answered that they were unsure 
about the extent of their commitment. One such respondent put it this way: “I’d be interested, but 
I am not sure I got a full feel for this.” The remaining 12 expressed their commitment. Of these, 
three noted that their commitment came from the sense of renewal they received at each project 
event, while two others cited the value of learning to their sense of commitment. One of these 
added, “I feel that the knowledge I’ve acquired will be beneficial to my school and as a student 
leader, I’m obligated to share it.” Four respondents described their various motivations for 
remaining committed, including becoming “excited about school improvement at all levels” and 
feeling “a need to explore and implement every feasible technique to foster an improved educational 
environment for our learners.” Three other replies could not be categorized. One participant wrote 
of remaining committed because of a desire to “continue with the positive effects it [Quest] has 
already had.” The value of collaboration was cited by another respondent, while a third wrote 
somewhat ambiguously, “ongoing process,” indicating perhaps the Quest project, continuous 
improvement, or commitment to such an endeavor. 

Goal Six: Commit to Continue the Quest Back Home 

With a mean rating of 4.32 and the lowest standard deviation of .55, participants reported 
with some consistency that the goal of committing to continue the Quest in their schools had been 
met. 



The ways in which attendees felt committed to continue school improvement efforts back at 
their schools were solicited in a final prompt on the open-ended feedback form. While seven 
respondents did not reply to this question, 19 answered that they were committed. Of these, ten 
offered generic descriptions of their commitment. Examples include, “we will take back the 
feedback and try to implement the suggestions,” “will try activities learned at Quest,” and 
“committees and changes.” One such respondent offering a generic reply wrote of being committed 
“if I am able to move the principal in this direction.” Nine respondents provided more specific 
descriptions of their commitments. “We will continue to use the Protocol,” reported one attendee, 
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while another planned to institute “AP courses, walkabouts, student club schedule, cross-curricular 
activity.” A twentieth respondent, however, expressed her uncertainty about commitment to continue 
school improvement efforts: “I do not really know how to answer these questions. I need more time 
to ‘process’ what has been introduced here. I am committed to the Quest for continuous school 
improvement at my school, but how receptive the ‘powers that be’ will be, I do not know.” It is 
possible that this respondent was one of the ten participants new to the Quest network. 

In an impromptu interview, one participant voiced a concern similar to that noted above. Of 
the rally content, the interviewee said, “It’s interesting, but I think the issue is will we be allowed 
to do some of the things we’ve learned here.” Although later in the interview she praised 
administrators at her school, the interviewee remained skeptical of their commitment to certain 
improvement efforts. It appears that, at least for a few network members, Quest offers useful 
information and insight whose use is nonetheless subject to endorsement by school administrators. 

On the other hand, the team presenting the Protocol process appeared committed to their use 
of it, offering various accounts of ways in which it had enhanced their sense of “professionalism.” 
And when other teams were asked how the process might be valuable in their schools, all identified 
potential uses. Two team members’ commitment to taking the Quest for school improvement back 
to their school was shared on the second day of the rally, when they described having brainstormed 
the night before a list of strategies for enhancing parent involvement. 



Comparative Findings 

Quantitative data from earlier evaluation reports (Howley-Rowe, 1999, 1998c) can be 
extracted to compare respondents’ rating of the six rally goals across Quest events. Such 
comparisons will provide a means to evaluate the extent to which Quest events consistently meet 
project objectives. 

It is interesting to note that mean ratings and standard deviations for the achievement of rally 
goals have changed very little over time (see Table 2). In fact, the mean rating of Goal One remained 
virtually identical from October 1997 to February 1999. 

However, it should be noted that both high school and elementary network members 
participated in the November 1998 rally (Howley-Rowe, 1999). One reason the November 1998 
ratings are somewhat higher than other high school ratings may be due to the influence of elementary 
network participants, who have tended to rate the achievement of rally goals more highly than high 
school participants. Unfortunately, quantitative ratings were not disaggregated by school level. 
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Table 2 



Mean Ratings of Achievement of Goals Across Rallies 





October 1997 


November 1998 


February 1999 




N 


Mean 


SD* 


N 


Mean 


SD* 


N 


Mean 


SD* 


Goal One 


28 


4.64 


.62 


37 


4.65 


.68 


28 


4.64 


.56 


Goal Two 


28 


4.46 


.79 


37 


4.51 


.56 


28 


4.29 


.71 


Goal Three 


28 


4.36 


.78 


37 


4.51 


.73 


28 


4.39 


.63 


Goal Four 


27 


4.41 


.69 


37 


4.51 


.61 


27 


4.33 


.68 


Goal Five 


28 


4.57 


.63 


37 


4.51 


.65 


28 


4.29 


.76 


Goal Six 


28 


4.46 


.69 


37 


4.81 


.46 


28 


4.32 


.55 



* Standard Deviation 



Goal Three, connecting with concepts related to continuous school improvement, increased 
very slightly by .03 when comparing the October 1997 and February 1999 rallies, while the mean 
rating for Goal One, connecting with colleagues, remained stable. Four mean ratings for 
achievement of goals at the February 1 999 rally had declined in comparison to October 1 997 ratings: 
Goal Four decreased by .08, Goal Six by .14, Goal Two by .17, and Goal Five by .28. These 
differences in mean ratings are so minimal as to have little practical significance, however. Also, 
standard deviations of ratings of achievement of goals fluctuated across rallies, although they were 
all .79 or less on the 5-point scale. 
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CONCLUSIONS AND RECOMMENDATIONS 



Conclusions 

Several conclusions may be drawn from the data presented in the previous section. 

The goal of connecting with colleagues, Goal One, appears to be the goal best achieved at 
the rally. Such connection occurred both in a social sense and in the sense that ideas were discussed. 
Thus, it can be concluded that Quest staff have been most successful in assisting Quest high school 
rally participants connect with each other. 

Interestingly, the two goals concerning the Quest learning community were least well- 
achieved, although as the ratings of both show, even these were deemed to have been achieved. Goal 
Two, creating a learning community, and Goal Five, committing to continue learning with the Quest 
learning community, both received the same mean rating of 4.29. Thus, it may be concluded that 
creating and sustaining commitment to Quest poses problems, although none apparently 
overwhelming, for some participants. One reason for the lower ratings of the achievement of these 
two goals may be that 1 0 of the 36 participants had not previously attended any Quest events, many 
of whom were students newly invited and school faculty assisting with the demonstration of the 
Protocol process. Of these, several also indicated that they had not known of the project prior to 
their participation in the February rally. 

Connecting with concepts concerning ongoing school improvement, Goal Three, seems to 
have been achieved. The achievement of this goal received the second highest mean rating (4.39), 
and respondents were able to cite specific content with which they had connected. Information about 
brain-based learning and the Protocol process were cited as particularly useful by attendees. 

Goal Four, creating personal and shared meaning, appears to have presented the most 
conceptual difficulties for participants. For example, the mean rating of 4.33 suggests that 
respondents thought this goal had been well achieved. However, non-responses and replies 
indicating confusion regarding the goal suggest that either the goal is unclear or that communicating 
what meaning was created is difficult for many participants. 

The sixth goal, committing to continue the Quest back home, was also achieved, with a mean 
rating of 4.32. Although many respondents expressed a high level of commitment to do so, a few 
felt their commitment faced some challenges once they returned to school. 

Achievement of rally goals appears to be fairly stable across rallies. Mean ratings of Goal 
Five, committing to continue learning with the Quest community, declined more steeply than ratings 
for other goals, although even this decline was minimal at .28. One can conclude that Quest rallies 
continue to be structured such that all goals are well met, albeit some more than others. 
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Recommendations 

The following recommendations for consideration during the design of future Quest high 
school rallies are made based upon analysis of data. 

First, Quest staff may want to be attentive to the ratings given for the achievement of goals 
concerning the Quest learning community. Although the ratings are not low relative to the whole 
scale, they were the lowest-rated. Staff should consider continuing to plan activities or means of 
communication aimed at enhancing the network community. Some such efforts have already been 
instituted, including financial assistance for those schools committing in writing to continue with 
Quest and dissemination to network members of the Quest Log (a project chronicle). 

A second recommendation is that Quest staff reconsider the way in which Goal Four, to 
create personal and shared meaning, is facilitated and communicated to high school network 
members. Although mean ratings indicated that attendees thought the goal had been achieved at the 
rally, many either did not respond or noted their uncertainty when asked to describe what meanings 
had been generated. Staff could consider offering examples to participants of what such meanings 
might be, or they could rephrase the goal entirely. Alternatively, staff could provide network 
members a more detailed rationale for and explanation of this goal in order to contextualize it for 
those who express confusion. Another alternative would be to structure reflective activities 
explicitly to enable participant articulation of meanings made during rally events. 

Because new participants from network schools continue to attend events, project staff 
should continue their efforts to orient new members to the Quest framework, methods, and goals. 
In addition, staff might request that veteran members of the network become mentors for new 
participants at project events in order to assist with orientation efforts. 

A fourth recommendation is for Quest staff to continue their efforts to support school 
improvement endeavors in network schools. Co- ventures, or site visits, are one way staff are already 
offering on-site advocacy. Other avenues might include weekly phone conversations with those 
schools experiencing particular difficulty implementing improvement activities or mailed updates 
concerning technical assistance resources. 

A methodological recommendation is for Quest staff to disaggregate data by school level 
should another combined elementary and high school rally be offered. This will assist staff in 
understanding more fully the similarities and differences between the two networks. 
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Feedback Forms 
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Inquiry Into Improvment 

Feedback Form 



The rally planners would appreciate your comments based upon the first day’s experience. 
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Quest High School Network Rally 
February 15-16, 1999 



Please describe how the following “worked” for you? 



Demonstration of Protocol 


Review of school improvement plans 


Review of brain-based learning research 


Use of Quest Instrument for Continuous School 
Improvement: Self-Scoring Profile 



Now, using the scale below, please circle the number that best describes the degree to which each 
of the following six goals were accomplished during the rally. 



5=Extremely well accomplished 3=Average l=Not accomplished 



Goal 1 : To connect with colleagues 

Goal 2: To create a learning community 

Goal 3: To connect with concepts concerning 
continuous school improvement 

Goal 4: To create personal and shared meaning 

Goal 5: To commit to continue learning with 
this community 

Goal 6: To commit to continue the Quest back 
home 



5 4 3 2 1 

5 4 3 2 1 

5 4 3 2 1 

5 4 3 2 1 

5 4 3 2 1 

5 4 3 2 1 
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Quest High School Network Rally 
February 15-16, 1999 




Quest staff continue to be interested in your experience at Quest rallies. Won V you please take a 
few minutes to complete the following questions? Your responses will remain anonymous and 
confidential 50 please feel free to be as candid_ as possible . Thank you! 



1 . In what capacity are you attending this rally? Please check one. 

Teacher Community member Other (please specify) 

Parent School administrator 

Student Support staff 

2. How many Quest events have you attended prior to this rally? 

3. In what ways do you feel that you connected with colleagues at this rally? 



4. Did Quest network members become more of a learning community during this rally? Give 
specific examples to support your answer. 



5. Describe examples of specific content related to continuous school improvement with which 
you “connected” at the rally. 



(over) 



6. If you were able to create personal and shared meanings at this rally, what were 2 or 3 of 
these meanings? 



7. To what extent do you feel committed to continuing to learn with the Quest community? For 
what reasons do you feel this? 



8. In what ways are you committed to continuing the Quest for continuous school improvement 
back at your school? 
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Citation Form 



I 

I 

I 



\e Program Evaluation Standards (1994, Sage) guided the development of this (check one): 

request for evaluation plan/design/proposal 

evaluation plan/design/proposal 

evaluation contract 

X evaluation report 
other: 



t To Interpret the information provided on this form, the reader needs to refer to the full text of the standards as they appear in Joint 
immittee on Standards for Educational Evaluation, The Program Evaluation Standards (1994), Thousand Oaks, CA, Sage. 

e Standards were consulted and used as indicated in the table below (check as appropriate): 



i 

r 



U2 



B 

U5 

I 

K 

P2 

i 

i 

A1 

R 

A4 

K 

i 

All 

I 

i 



criptor 

Stakeholder Identification 


The Standard was 
deemed applicable 
and to the extent 
feasible was taken 
into account. 

X 


The Standard was 
deemed applicable 
but could not be 
taken into account. 


The Standard was 
not deemed appli- 
cable. 


Exception was taken 
to the Standard. 


Evaluator Credibility 


X 








Information Scope and Selection 


X 








Values Identification 


X 








Report Clarity 


X 








Report Timeliness and Dissemination 


X 








Evaluation Impact 


X 








Practical Procedures 


X 








Political Viability 






X 




Cost Effectiveness 


X 








Service Orientation 


X 








Formal Agreements 


X 








Rights of Human Subjects 


X 








Human Interactions 


X 








Complete and Fair Assessment 


X 








Disclosure of Findings 


X 








Conflict of Interest 


X 








Fiscal Responsibility 


X 








Program Documentation 


X 








Context Analysis 


X 








Described Purposes and Procedures 


X 








Defensible Information Sources 


X 








Valid Information 


X 








Reliable Information 


X 








Systematic Information 


X 








Analysis of Quantitative Information 


X 








Analysis of Qualitative Information 


X 








Justified Conclusions 


X 








Impartial Reporting 


X 








Metaevaluation 


X 
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